MUC/MET Evaluation Trends

A L ENTITY TASK ( MET ) OVERVIEW Roberta Merchant

In November, 1996, the Message Understanding Conference-6 (MUC-6) evaluation of named entity identification demonstrated that systems are approaching human performance on English language texts [10]. Informal and anonymous, the MET provided a new opportunity to assess progress on the same task in Spanish, Japanese, and Chinese. Preliminary results indicate that MET systems in all three language...

The Multilingual Entity Task (MET) Overview

In November, 1996, the Message Understanding Conference-6 (MUC-6) evaluation of named entity identification demonstrated that systems are approaching human performance on English language texts [10]. Informal and anonymous, the MET provided a new opportunity to assess progress on the same task in Spanish, Japanese, and Chinese. Preliminary results indicate that MET systems in all three language...

MUC-4 evaluation metrics

The MUC-4 evaluation metrics measure the performance of the message understanding systems . This paper describes the scoring algorithms used to arrive at the metrics as well as the improvements that were made to th e MUC-3 methods . MUC-4 evaluation metrics were stricter than those used in MUC-3. Given the differences in scoring between MUC-3 and MUC-4, the MUC-4 systems' scores represent a lar...

CRL's Approach to MET

From February to April CRL carried out investigations into the modification of our l~n~Hsh name recognition software developed for MUC-6 [1] to Chinese and Spanish. In addition a Japanese system, developed under Tipster Phase I [2], was modified to comply with the MET task. Finally learning methods developed for MUC-6 were adapted to handle Chinese. All systems performed with good levels of acc...

Description of the Oki System as Used for MET-2